Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness

Probabilistic forecasts, calibration and sharpness

Probabilistic forecasts of a continuous variable take the form of predictive densities or predictive cumulative distribution functions. We propose a diagnostic approach to the evaluation of predictive performance that is based on the paradigm of maximizing the sharpness of the predictive distributions subject to calibration. Calibration refers to the statistical consistency between the distribu...

Generalised Density Forecast Combinations *

Density forecast combinations are becoming increasingly popular as a means of improving forecast ‘accuracy’, as measured by a scoring rule. In this paper we generalise this literature by letting the combination weights follow more general schemes. Sieve estimation is used to optimise the score of the generalised density combination where the combination weights depend on the variable one is try...

Evaluating density forecasts: a comment

EVALUATING PROBABILITY FORECASTS: Calibration Isn’t Everything

Abstract: Using evaluation methodologies for rare events from meteorology and psychology, we examine the value of probability forecasts of real GDP declines during the current and each of the next four quarters using data from the Survey of Professional Forecasters. We study the quality of these probability forecasts in terms of calibration, resolution, the relative operating characteristic (RO...

Evaluating Density Forecasts via the Copula Approach

In this paper, we develop parametric tests for the correct density forecasts. Similar to Berkowitz (2001), we construct our tests by nesting a series of i.i.d. uniform random variables in a class of stationary Markov processes. Unlike Berkowitz (2001), the class of Markov processes in this paper is constructed via the copula approach, which allows the separate modeling of the marginal distribut...